
Milwaukee Longitudinal School Choice 
Evaluation: Annual School Testing 
Summary Report 2007-08 



Nathan L. Gray 
Patrick J. Wolf 
Laura I. Jensen 

SCDP Milwaukee Evaluation 



Report #9 
March 2009 



The University of Arkansas 

was founded in 1871 as the flagship institution 
of higher education for the state of Arkansas. 
Established as a land grant university, its mandate was threefold: to teach students, conduct 
research, and perform service and outreach. 

The College of Education and Health Professions established the Department of Education 
Reform in 2005. The department’s mission is to advance education and economic development 
by focusing on the improvement of academic achievement in elementary and secondary schools. 
It conducts research and demonstration projects in five primary areas of reform: teacher quality, 
leadership, policy, accountability, and school choice. 

The School Choice Demonstration Project (SCDP), based within the Department of Education 
Reform, is an education research center devoted to the non-partisan study of the effects of school 
choice policy and is staffed by leading school choice researchers and scholars. Led by Dr. Patrick 
J. Wolf, Professor of Education Reform and Endowed 21st Century Chair in School Choice, 
SCDP’s national team of researchers, institutional research partners and staff are devoted to the 
rigorous evaluation of school choice programs and other school improvement efforts across the 
country. The SCDP is committed to raising and advancing the public’s understanding of the 
strengths and limitations of school choice policies and programs by conducting comprehensive 
research on what happens to students, families, schools and communities when more parents are 
allowed to choose their child’s school. 
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Executive Summary 

With the passage of the 2005 Wisconsin Act 125, private schools participating in the Milwaukee Parental 
Choice Program (MPCP) are now required to administer a nationally normed standardized test annually 
in reading, mathematics, and science to their MPCP (a.k.a. “Choice”) students enrolled in the 4 th , 8 th , 
and 10 th grades. The law further directs Choice schools to submit copies of the scores from those tests to 
the School Choice Demonstration Project for processing and reporting to the Legislative Audit Bureau. 
During the 2007-08 school year, MPCP schools administered either nationally normed tests, such as the 
Iowa Test of Basic Skills, or the criterion referenced Wisconsin Knowledge and Concepts Examinations 
(WKCE). The School Choice Demonstration Project (SCDP) received student test scores from 114 of 
the 115 schools participating in the MPCP that were required to administer tests. Specifically, the SCDP 
received 5,763 nationally normed student test scores from 81 schools and 1,735 WKCE scores from 36 
schools. These school numbers add to 123 because some schools used both types of exams and sent all 4 th , 
8 th , and 10 th grade scores to the SCDP. 

The average nationally normed test scores received by the SCDP from the Choice schools ranged 
from the 33 rd to near the 41 st percentiles, depending on the grade level and subject area. Such levels of 
performance are typical for the population of low-income urban students served by the MPCP. On 
average, 4 th grade students in the MPCP scored the lower-third of the performance distribution in 
all three subject areas. The Choice students tested in 8 th grade performed near the 33 rd percentile. The 
MPCP students tested in 10 th grade, on average, performed slightly better than their 8 th and 4 th grade 
counterparts, from the 35 th percentile in math and science to near the 41 st percentile in reading. 

When the set of 4 th , 8 th , and 10 th grade MPCP test scores from 2007-08 in reading, math, and science 
are compared to the scores provided to the SCDP in 2006-07, the average scores by grade are higher for 
the MPCP students in 2007-08 for eight of the nine comparisons. Since every year a different group of 
MPCP students takes the accountability tests, we caution readers against drawing firm conclusions from 
this comparison of cross-sectional data across years. 

The performance of the Choice students who took the criterion referenced WKCE can be approximately 
compared to that of similarly income-disadvantaged students in Milwaukee Public Schools (MPS) who 
took the same test. Students defined as income disadvantaged are those participating in the free and 
reduced price lunch program (FRL). Henceforth, income-disadvantaged from MPS will be referred to as 
“MPS FRL.” The 4 th grade MPCP students who took the WKCE on average scored 2 to 14 scale score 
points (equal to .17 to .36 of a standard deviation) below the average scores of MPS FRL 4 th graders. The 
8 th grade MPCP students who took the WKCE, however, performed better than MPS FRL students by 
1 to 7 scale score points (.02 to .18 of a standard deviation). Only 103 Choice students in 10 th grade took 
the WKCE, too few to generate reliable performance results. 
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Readers are strongly urged not to draw conclusions about the relative performance of the MPCP or MPS 
based upon these rough descriptive comparisons. The differences in average test scores observed at the 
4 th and 8 th grade levels could be explained by the different types of students who self-sort into the MPCP 
or MPS. Moreover, the subset of MPCP schools that administer the WKCE as their accountability test 
may not be representative of the total population of MPCP schools. Any reliable determination of the 
effectiveness of a school choice program like the MPCP can only come from a rigorous experimental or 
longitudinal study that follows a representative group of choice students over time and compares their 
achievement gains to those of a comparable set of public school students. For such an evaluation we refer 
readers to the MPCP Longitudinal Educational Growth Study (LEGS) also being conducted by the 
SCDP 



This ongoing research project is being funded 
by a diverse set of philanthropies including the 
Annie E. Casey, Joyce, Kern Family, Lynde and 
Harry Bradley, Robertson, and Walton Family. 

We thank them for their generous support and 
acknowledge that the actual content of this report 
is solely the responsibility of the authors and does 
not necessarily reflect any official positions of the 
various funding organizations or the University of 
Arkansas. 




This report and its companion reports are the second in a series of annual reports on the Milwaukee 
Parental Choice Program (MPCP) that will be conducted by the School Choice Demonstration Project 
(SCDP). An initial draft of this report was 
greatly improved based on comments from the 
SCDP Research Advisory Board, particularly 
Anneliese Dickman of the Public Policy Forum and 
Margaret Raymond of the Hoover Institution at 
Stanford University. Any remaining errors are the 



responsibility of the authors alone. 
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Introduction 



On March 10, 2006, Wisconsin Governor Jim Doyle signed Wisconsin Act 125 into law. The Act 
modified several elements of the Milwaukee Parental Choice Program (MPCP), the nation’s first and 
largest urban school voucher program. Participating private schools are now required to administer a 
nationally normed standardized test annually in reading, mathematics, and science to their MPCP (a.k.a. 
“Choice”) students enrolled in the 4 th , 8 th , and 10 th grades. Beginning in 2006 and extending through 
2011, the individual student results of the tests must be provided to the School Choice Demonstration 
Project (SCDP). Finally, Act 125 requires that: 

The [Wisconsin] legislative audit bureau shall review and analyze the 
standardized test data received from the School Choice Demonstration Project. 
Based on its review, in 2007 and annually thereafter until 2011, the bureau 
shall report to the legislature under s. 13.172 (2) the result of the 
standardized tests administered under subd. 1., the scores of a representative 
sample of pupils participating in the program under ss. 118.30 and 121.02 (1) 
(r), and scores of a comparable group of pupils enrolled in the school district 
operating under this chapter on the tests under ss 118.30 abd 121.02 (1) (r). 1 

This report describes the results of the student testing conducted by MPCP schools during the 2007-08 
academic year. 2 The standardized test scores were collected from participating schools throughout the 
school year, with most of them arriving at the SCDP in the summer of 2008. The SCDP staff carefully 
entered these scores into a single database and delivered the data to the Legislative Audit Bureau on 
January 6 th , 2009. 3 

Most of the test scores received from Choice schools were based on the administration of nationally 
normed tests such as the Terra Nova or the Iowa Test of Basic Skills (ITBS). These types of tests measure 
performance relative to other students by including questions meant to produce a full range of scores 
(i.e. very easy questions ranging to very difficult questions to separate the highest and lowest performing 
students). In contrast, 36 private schools participating in the MPCP administered the Wisconsin state 
criterion referenced test — the Wisconsin Knowledge and Concepts Examinations (WKCE) (Table 1). 
Even though the producers of both norm and criterion referenced tests claim that the assessments cover 



1 Wl Act 1 25, Sec. 8, 1 1 9.23 (7)(e), 2. 

2 Although this report is mandated by Act 1 25, the law specifies that it be conducted by an independent research 
organization (i.e. the SCDP) and financed by non-governmental sources. 

3 The majority of MPCP schools administered the standardized tests late in the spring. The companies that produce the 
tests then require several months to score them and send the test results to the schools. Since most schools operate 
with a minimum staff over the summer, in some cases the test results were not compiled and sent to the SCDP until the 
fall of 2008. The last set of test scores from an MPCP school was received on October 29, resulting in an unexpected 
delay in data input and delivery of the test score database to the Legislative Audit Bureau. 
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the same subject domain, students likely exhibit different patterns of proficiency on the two types of tests 
due to differences in framework and item selection. The WKCE is only administered in Wisconsin; 
therefore, no national distribution exists to make a direct performance comparison with the ITBS and 
Terra Nova. 



To account for these differences, distinct subsections of this report present consolidated results from 
schools that administered the various nationally normed standardized tests and separate results from 
schools that administered the state-specific exam. For reference purposes, the results of the subgroups of 
Choice students and schools that administered the WKCE are compared with the overall and school-level 
results on the WKCE of Milwaukee Public School (MPS) students participating in the federal lunch 
program . 4 



Table 1 : Types of Tests Taken by MPCP Schools and Students 



Schools Students 

Number Percentage Number Percentage 



National Norm 


86 


69.9% 


5,763 


77.0% 


WKCE 


37 


30.1% 


1,735 


23.0% 


Total 


123 


100.0% 


7,498 


100.0% 



The scores received from the MPCP schools are summarized in two results sections below. The first 
results section describes the aggregate scores for the groups of Choice students in each of the tested grades 
who took either nationally normed assessments or the WKCE. The second results section describes the 
distribution of MPCP test scores, by grade and subject matter, at the individual school level. 

Although school-level test scores are presented in the second results section, individual schools are not 
identified by name. Connecting any individual Choice school explicitly to information about its students 
could enable readers to identify individual participants in the study in violation of the assurances of 



4 As a mechanism for comparing MPCP and MPS students, eligibility for the federal lunch program is limited in two ways. 
First, the family income ceiling for eligibility for the lunch program is 185 percent of the poverty line, which is slightly 
higher than the income ceiling of 175 percent of poverty for initial eligibility for the MPCP but somewhat below the 
income ceiling of 220 percent of poverty for renewal of MPCP eligibility. Second, many students who are income- 
eligible for the federal lunch program choose not to participate. The rate of non-participation tends to increase steadily 
as students move from the lower grades to the higher grades. Although federal lunch program participation is an 
imperfect measure of family disadvantage, it was the best criterion available to generate approximate comparisons 
for this particular element of the evaluation. For this and other reasons described in this report, readers are cautioned 
against drawing any strong conclusions about the relative performance of MPCP and MPS students from the descriptive 
comparisons provided here. 
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confidentiality that are required when conducting such research . 5 The preservation of the anonymity of 
participants in educational evaluations is so important that the federal statute establishing the evaluation 
division of the U.S. Department of Education expressly prohibits the naming of individual students, 
parents, or schools in any of its reports . 6 Because state law requires the affected MPCP schools to 
administer tests and submit scores to the SCDP, we do mention by name in this report the schools that 
did and did not perform those required actions (Appendix A). Because the information thus submitted is 
to facilitate an education evaluation, however, any subsequent presentation of the data provided by schools 
and students must remain anonymous. Of the hundreds of statistical studies of school choice programs 
with which we are familiar, none of them have revealed school-level information directly associated with 
named schools, for these very reasons . 7 

The MPCP Annual School Testing Summary Report has important strengths and limitations. The main 
strength of the Report is its ability to provide a data-rich snapshot of the current academic performance 
of a large number of students from nearly all of the schools participating in the MPCP. Such information 
on the Choice program has not been available for more than a decade. The limitations of the study stem 
largely from the fact that the data are cross-sectional, not longitudinal, and the two comparison groups 
involved are far from ideal. These data show us how well a large group of Choice students is performing 
academically, but tell us nothing about what has caused them to perform at that level. Since many 
factors including the backgrounds and home lives of students as well as the quality of their educational 
experiences likely influence their performance on standardized tests at a given point in time, it would be a 
mistake for readers to draw conclusions concerning the effectiveness of the MPCP based on these simple 
annual descriptive statistics. 



5 As with all academic research with human participants, the SCDP research team had to gain approval of an extensive 
protocol for protecting the anonymity of participants and the confidentiality of the information that they provide before 
research on the MPCP could begin. Approvals of our research protocols, which prohibit us from associating any data 
with named individuals or schools, were obtained from the Institutional Review Boards for Human Subjects Research at 
the University of Arkansas, the University of Wisconsin, the University of Kentucky, and Westat. 

6 "The Director shall ensure that all individually identifiable information about students, their academic achievements, 
their families, and information with respect to individual schools, shall remain confidential... "See Education Sciences 
Reform Act of 2002, 20 U.S.C., 1 232g, 1 232h. 

7 Many schools in the MPCP provide school-level information by school name voluntarily to parents and organizations to 
facilitate the school choice process. In Milwaukee and in other major cities around the country, the organization Great 
Schools Dot Net is spearheading efforts to consolidate such information into comprehensive school choice guides 
called "My School Chooser." Because such efforts are voluntary and are not part of a research evaluation, the prohibition 
against connecting descriptive information to named schools does not apply to such school choice guides. Copies of 
the "Milwaukee School Chooser"are available at: http://www.greatschools.net/geo/landing/milwaukee.page. 



Milwaukee Longitudinal School Choice Evaluation: Annual School Testing Summary Report 2007-08 





6 



March 2009 



The Longitudinal Educational Growth Study, also being conducted by the SCDP, will entail a comparison of 
the achievement gains over time of a representative sample of MPCP students relative to a carefully- matched 
set of peers in Milwaukee traditional public and charter schools. It will track the performance of the same 
set of MPCP and MPS students as they progress through their educational experience. The MPCP Annual 
School Testing Summary Report, in contrast, examines a different set of MPCP students each year at fixed 
points in their educational experience. It is not a growth or value added comparison against peers in MPS or 
any other group. This point cannot be stressed enough. Readers who are interested in student performance 
differences that can be reasonably attributed to the influence of the Choice program itself are advised to follow 
the progress of the MPCP Longitudinal Educational Growth Study. 

Before we present the results of the MPCP student testing for 2007-08, however, we first explain how the 
MPCP test scores were acquired. 



Process for Obtaining MPCP Test Scores 




The School Choice Demonstration Project engaged in an extensive set of activities during the 2007-08 
academic year that culminated in this report. On August 21 st , 2007, Dr. Patrick Wolf, principal investigator 
for the SCDP, sent a letter and the MPCP Longitudinal Evaluation FAQ_to the principal of each school 
participating in the MPCP reminding them of the testing requirements mandated by Wisconsin Act 125 
(Exhibit 1). On September 11 th , 2007, Dr. Wolf, along with several additional members of the SCDP research 
team, traveled to Milwaukee to attend the first Pupil Assignment Council (PAC) meeting of the 2007-08 

school year. During the PAC meeting, Dr. 

Wolf gave a presentation to the MPCP school 
administrators again reminding them of the 
accountability testing requirements of Act 125. 



On February 5 th , 2008, representatives of the 
SCDP attended the 3rd PAC meeting of the 
2007-08 school year. Following a presentation 
reminding the school administrators of the 
mandates ofWisconsin Act 125, letters and 
pre-labeled FedEx packages were distributed for 
the schools to use when mailing accountability 
test scores to the SCDP (Exhibit 2). The 
February 5 th letter provided instructions for 
using the FedEx packages as well as a time 
frame for submitting the test scores. The school 
administrators were notified that scores from 
the fall 2007 test should be mailed to the SCDP 
by March 1 st , 2008, and that scores from spring 
2008 testing should be mailed to the SDCP 
by July 1 st . In early February, SCDP research 
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assistants did follow-up mailings of the FedEx packages to schools that did not send representation to the 
February 5 th PAC meeting. 

On July 15 th , 2008, a reminder letter was mailed to principals of the 25 MPCP schools that had yet 
to send in test scores from the 2007-08 school year (Exhibit 3). Through subsequent contact with the 
remaining schools, the SCDP received test scores from an additional 15 schools. Thus, by mid-August, 
105 of the 115 MPCP schools that were required to test had submitted their scores. 

On August 13 th , 2008, Dr. Wolf notified officials at the Wisconsin Department of Public Instruction 
(DPI) responsible for overseeing the MPCP of the 10 schools that had failed to comply with the 2007- 
OS testing requirements. The schools were subsequently contacted by the DPI and informed that they 
would not be receiving further voucher payments until they supplied the SCDP with 2007-08 test scores 
for their students. Nine of the 10 schools submitted their scores, with the final set of scores received on 
October 29, 2008. In this second year of student testing under Act 125, the SCDP received test scores 
from 114 of the 115 affected schools, a 99 percent response rate (Appendix A). 

Use of Percentile Rankings 

Because private schools administered several types of tests, as permitted by Act 125, it is necessary to use 
the standardized version of the student scores, National Percentile Ranks (NPRs or national percentiles), 
in order to combine and compare them. Scale scores often are the preferred metric for reporting test 
scores because they are produced by applying a complex scoring system, called Item Response Theory, 
which draws from the entire pattern of correct and incorrect answers and factors in different levels of 
difficulty. Scale scores are “vertically equated,” meaning that scores on the same test can be compared 
across grades. However, scale scores are not “horizontally equated” in anyway across different tests. 

Even for norm-referenced tests such as the ITBS and the Terra Nova, the scale scores for corresponding 
national percentiles are drastically different. For example, for the 4 th grade test, the corresponding scale 
score for the 50 th percentile on the ITBS is 200. For the 4 th grade Terra Nova, the 50 th percentile scale 
score is 637. Clearly, the scale scores are inappropriate for averaging across tests. Therefore, for all 
nationally-normed standardized tests, NPRs are used in this report. All participating schools from which 
norm referenced scores were retrieved reported student NPRs for reading, math, and science. These scores 
were consolidated to produce the aggregate performance totals provided below. 8 

In the case of the WKCE, no national norm exists and thus no national percentile exists. Therefore, 
this report presents the school level data in terms of scale scores for the subset of MPCP schools that 



8 Although all of these scores are similar in that they describe the student's performance in comparison with the national 
sample of students that took the test (i.e. the "norming" population), that national sample can vary somewhat across the 
tests - another reason why readers are cautioned against drawing strong conclusions from these illustrative data. 
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administered the WKCE. Because MPS uses the WKCE, we provide information about the performance 
of MPS FRL students to provide a general context in which to interpret the scale score performance levels 
of the MPCP students and schools that administered the WKCE as their accountability test. 



Percentile Results at the Program/Grade/Subject Level 

The following table 9 reports the information contained in these data concerning the NPR averages for 
Choice students in grades 4, 8, and 10 in the subjects of reading, math, and science for the two years 
of data which the SCDP has received. Readers are cautioned about comparing one year’s results with 
another as different students took these tests in different years. These results are not longitudinal in 
nature. 



Table 2: 2006-07 and 2007-08 National Percentile Summary Statistics 







Read 


2006-07 

Math 


Science 


Read 


2007-08 

Math 


Science 


4th Grade 


Mean 


34.1 


32.3 


28.0 


37.8 


35.0 


32.9 




Obs 


564 


572 


321 


654 


648 


701 




Std. Dev. 


25.2 


25.0 


22.7 


25.5 


24.8 


24.3 



8th Grade 



Mean 


35.9 


33.2 


28.6 


36.3 


36.1 


33.8 


Obs 


407 


405 


291 


443 


436 


485 


Std. Dev. 


24.6 


26.3 


21.7 


25.2 


25.8 


22.3 



10th Grade 



Mean 


38.4 


34.3 


38.8 


40.7 


35.1 


35.2 


Obs 


463 


447 


446 


508 


531 


518 


Std. Dev. 


25.5 


22.9 


25.8 


25.8 


22.4 


25.4 



9 Including 2006-07 in the table is for reference only. Readers should not compare one year's scores with another year's 
scores as different students took the tests that produced these results. 
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In 2008, the SCDP received scores produced by 1,215 4 th grade MPCP students. 10 These students 
on average scored around the lower- third of the national performance distribution in all three subject 
areas. These results are not surprising. The MPCP is limited to low-income students, a disadvantaged 
population that typically exhibits test scores well below the national average. These particular scores do 
not indicate how well the program serves these students; they merely describe the average performance of 
an educationally disadvantaged group of students at a particular moment in time relative to the average 
student in the nation, most of whom are not so disadvantaged. To interpret the results, one would say 
that the 4th grade Milwaukee Choice students taking a nationally standardized test averaged near the 
38 th percentile (scored higher than 38 percent) in reading compared to other 4 th graders nationwide taking 
similar tests. Choice students in the 4th grade averaged in the 35 th national percentile in math and the 
33 rd national percentile in science. 

The MPCP students tested in 8 th grade performed similarly (Table 2). The SCDP received scores 
generated by 1,116 8 th graders in the MPCP. Choice students in 8 th grade, on average, performed around 
the lower-third of the national distributions in all three subject areas. 

The 508 MPCP students tested in 10 th grade scored the highest relative to national norms of the three 
grade-cohorts tested (Table 2). The 10 th grade Choice students scored near the 41 st percentile in reading 
and the 35 th percentile in math and science. Although these descriptive statistics appear to show some 
academic improvement as Choice students mature, these grade-cohorts of students are compositionally 
different. Readers are cautioned against inferring from these data that the MPCP is responsible for the 
difference in performance between 10 th grade MPCP students and their 8 th and 4 th grade counterparts. 

Scale Score Results at the Program/Grade/Subject Level 

The fact that a subset of MPCP students as well as MPS students took the WKCE criterion referenced 
test allows us to present those results in a different manner. MPS FRL students are likely to be more 
similar to MPCP students than any national norming population, since both groups of students live in 
the same city and qualify as low income. Still, because this approximate match is not very precise, and 
because the subset of MPCP students who took the WKCE is not likely to be representative of all MPCP 
students, readers are cautioned against drawing conclusions about the effects of the Choice program from 
this limited snapshot comparison. 



10 The total number of student test scores by grade reported here and throughout the text are counts of the number of 
different students for whom at least one test score on a norm-referenced test or the WKCE was provided. That number 
for each grade is higher than the "Observations" numbers reported in each table because we separate out the WKCE 
scale scores from the norm-referenced percentile scores and because most but not all of the students produced test 
scores in all three of the subject areas. 
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Table 3 illustrates the summary statistics for 4 th and 8 th grade MPCP students who took the WKCE in 
the fall of 2008 as well as the statistics from similarly MPS FRL students. Only 103 MPCP 10 th graders 
took the WKCE last year, making aggregate statistics about that small subgroup insufficiently reliable to 
present here. These descriptive statistics show 4 th graders in the MPCP who took the WKCE performing 
9 to 18 scale points below the average levels of MPS FRL 4 th graders. This achievement difference 
is equal to .17 to .36 of a standard deviation of the MPS test score distribution. The 8 th grade scores 
illustrate the opposite. The scale scores for the MPCP students in 8 th grade on the WKCE are higher in 
all three subjects, by 1 to 7 scale score points, than those of similarly MPS FRL 8 th grade students. This 
achievement difference at the 8 th grade level is equal to .02 to .18 of a standard deviation. For both grades 
and in all three subjects the differences in the average scale scores of the two groups are relatively modest 
in size. 



Table 3: 2006-07 and 2007-08 WKCE Summary Statistice for Scaled Scores Student Level 



Reading Math Science 

MPCP MPS MPCP MPS MPCP MPS 





06-07 


07-08 


06-07 


07-08 


06-07 


07-08 


06-07 


07-08 


06-07 


07-08 


06-07 


07-08 


4th Grade 


























Mean 


436 


429 


444 


438 


417 


411 


430 


429 


264 


272 


273 


272 


Obs 


338 


457 


4654 


4723 


338 


465 


4811 


4825 


338 


409 


4673 


4810 


Std. Dev. 


50 


56 


48 


51 


20 


53 


47 


50 


31 


61 


31 


29 



8th Grade 



Mean 


494 


486 


485 


480 


503 


489 


497 


488 


365 


368 


359 


361 


Obs 


401 


579 


4834 


4641 


402 


581 


4917 


4657 


399 


561 


4779 


4595 


Std. Dev. 


53 


52 


52 


55 


44 


53 


47 


54 


42 


51 


41 


38 



Performance Distributions at the Individual School Level 




This section presents a series of histograms that describe the 
distribution of results of the 2007-08 MPCP school testing at 
the school level. The histograms illustrate 4 th and 8 th grade scores 
in reading, math, and science. Figure 1 represents 4 th grade and 
Figure 2 shows 8 th grade in all three subjects. These histograms 
have a normal distribution overlay, which is depicted by the line 
in each graph. This normal distribution overlay is relative to the 
empirical data that underlay it. That means it is appropriately 
scaled and has the same mean and standard deviation as the 
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data. These histograms allow for slightly more disaggregation. For example, as mentioned earlier, the 4 th 
grade reading percentile mean was 37.8, indicating students on average ranked just above the lower one- 
third of the distribution. Looking at school averages in the 4 th grade reading histogram, however, there 
were 17 MPCP schools whose Choice students averaged higher than the 50 th percentile in their test score 
performance. 

These particular histograms use frequency counts of the number of schools with average percentile scores 
in each decile. For example, in Figure le, the Choice students in 14 schools had an average national 
percentile score in Grade 4 reading between the 50 th and 60 th percentiles. In general, these graphs are 
intended to indicate clusters of low-performing, average-performing, and high-performing MPCP 
students grouped at the school level. 

Figure 1 . 4th Grade National Percentile Rank for MPCP Schools 



Grade 4 National Percentile Reading Histogram 




Grade 4 National Percentile Math Histogram 




Grade 4 National Percentile Science Histogram 
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Figure 2. 8th Grade National Percentile Rank for MPCP Schools 



Grade 8 National Percentile Reading Histogram 



Grade 8 National Percentile Math Histogram 





Grade 8 National Percentile Science Histogram 




The histograms of MPCP school-level test scores presented above generally show distributions with 
what is known as a “positive skew.” The school-level performance averages tend to cluster around the 40 th 
percentile, with a small group of much higher performing units at the upper tail of the distribution. The 
positive skew of the distributions lessons somewhat in the upper grades and takes more of the shape of a 
normal bell-shaped curve. The distributions reveal that some school-level clusters of MPCP students are 
performing very well relative to national norms; however, most school-level clusters of students performed 
just below national averages. Because these data merely provide a snapshot of school-level groupings of 
students, we cannot infer from them that some MPCP schools are much better performing than most 
MPCP schools, since the results could be due to higher-performing groups of MPCP students gravitating 
towards particular MPCP schools. In other words, we cannot rule out student self-sorting as the cause of 
the school-level performance distributions presented above. 

Table 4 illustrates the WKCE scale score data aggregated to the school level. Although similar to Table 
3, we see how statistics change when test scores are distributed among schools and then averaged at 
the school level as opposed to aggregated across an entire program or school system. Under almost all 
circumstances, the averages of subgroup averages (e.g. performance by school) will provide different results 
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than taking the total average of the population (e.g. all testers). 11 Because a small number of MPCP 
schools are performing well above both the MPCP and MPS system-level averages, the WKCE scores 
averaged at the school level result in comparisons somewhat more favorable to the MPCP schools than 
the comparisons based on test-scores at the individual student level. Still, the performance of school-level 
groupings of 4 th grade Choice students remains lower than the performance of school-level groupings of 
4 th grade MPS FRL students. The range of standard deviation units for 4 th grade is -.62 to -.12. A similar 
comparison of school-level results in 8 th grade shows that MPCP student performance averaged at the 
school level is higher than that of 8 th grade MPS FRL students by 13 to 16 scale score points (.42 to .57 
of a standard deviation). 



Table 4: 2006-07 and 2007-08 WKCE Summary Statistice for Scaled Scores School Level 



Reading 


Math 


Science 


MPCP MPS 


MPCP MPS 


MPCP MPS 


06-07 07-08 06-07 07-08 


06-07 07-08 06-07 07-08 


06-07 07-08 06-07 07-08 



4th Grade 



Mean 


442 


433 


442 


439 


423 


416 


428 


430 


270 


271 


272 


273 


Obs 


26 


23 


129 


118 


26 


23 


130 


118 


26 


23 


128 


118 


Std. 

Dev. 


37 


34 


30 


18 


43 


40 


29 


19 


26 


41 


19 


12 



8th Grade 



Mean 


509 


494 


481 


478 


512 


Obs 


36 


27 


98 


90 


36 


Std. 

Dev. 


33 


34 


29 


28 


30 



498 


491 


485 


376 


365 


353 


358 


27 


98 


90 


36 


27 


98 


90 


38 


27 


29 


36 


23 


30 


21 



The following 12 histograms graphically show the Table 6 data for those MPCP schools and the MPS 
schools taking the WKCE. 



1 1 In statistical parlance, the fact that an average of subgroup averages rarely equals the average of the entire group is 
called the "ecological inference" phenomenon. 
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Figure 3. 4 th Grade WKCE Reading Histograms for MPCP and MPS Schools 



Grade 4 Choice WKCE Reading Histogram 




400 450 

Reading Scaled Score 



Grade 4 MPS WKCE Reading Histogram 




As evidenced in the above two graphs, the means of the two populations are almost identical, but clearly 
the MPCP group has a higher standard deviation (e.g. greater variation in performance) and has more 
schools in both the upper and lower tail of the distribution. We would expect this wider variation in 
school-level achievement among the variegated set of MPCP schools. 

Figure 4. 4 th Grade WKCE Math Histograms for MPCP and MPS Schools 



Grade 4 Choice WKCE Math Histogram 




Grade 4 MPS WKCE Math Histogram 




A similar story is true for the math scores, and the distribution appears to be most normal for this subject 
area. 
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Figure 5. 4th Grade WKCE Science Histograms for MPCP and MPS Schools 



Grade 4 Choice WKCE Science Histogram 




Grade 4 MPS WKCE Science Histogram 




The science scores have the highest standard deviation difference between the two groups yet the smallest 
mean difference. These differences are likely due to the one outlier in the MPCP histogram— one MPCP 
school averaged relatively high scores on the science exam. With the small number of schools in the 
MPCP program, these two higher averaging schools skew the mean and standard deviation results higher 
at the school level. These types of results are expected in such an analysis. 

Figure 6. 8 th Grade WKCE Reading Histograms for MPCP and MPS Schools 



Grade 8 Choice WKCE Reading Histogram 




Grade 8 MPS WKCE Reading Histogram 




In the case of 8 th grade reading, we see a relatively normal distribution for each of the two groups. The 
MPCP histogram shows a higher mean than the MPS figure, by 16 points, and there are more schools 
represented in the upper tail of the MPCP distribution than in the lower tail. 
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Grade 8 Choice WKCE Math Histogram 




Grade 8 MPS WKCE Math Histogram 




Grade 8 Choice WKCE Science Histogram 




300 350 40o" 

Science Scaled Score 



450 



Grade 8 MPS WKCE Science Histogram 




The math and science histograms for 8 th grade also show relatively normal distributions for each group. 
The 8 th grade math and science means in the MPS groups have been influenced by a spike near the mean 
which generates a more narrow distribution of school-level scores than experienced in the MPCP, which 
has no clear spike around the mean and a wider distribution of scores. 

These tables and histograms suggest there are schools in each group (MPCP and MPS FRL) in which 
students are performing well above or well below the MPS FRL group average. These diagrams suggest 
there is ample variation in both groups. The standard deviations for the MPCP group are higher, which is 
likely because of the lower number of schools and greater diversity of schools in that group. One should 
be careful in interpreting these data. The differences across the comparison groups and between the grades 
cannot necessarily be attributed to the Choice program. The figures are presented merely to provide a 
general description of the school-level performance of MPCP students and place that performance in 
context by making a rough comparison with the school-level performance of income disadvantaged MPS 
students. 
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Proficiency Score Results 

Scaled scores, however, are not the only way to report results from criterion referenced tests such as the 
WKCE. Often times, states and schools report scores in terms of proficiency rates. That is, if a student 
received a specific scaled score or higher, then the student is proficient in that particular subject area. This 
particular section of this report shows the MPCP students’ results in terms of the percent proficient. 
Table 5 shows the percent proficient in each grade for each subject for MPCP and the free and reduced 
lunch students of MPS for which we had sufficient data. 



Table 5: MPCP and MPS Percent Proficient 



Subject 


MPCP 


MPS FRL 


4th Grade Reading Percent Proficient 


46% 


55% 


4th Grade Math Percent Proficient 


29% 


45% 


4th Grade Science Percent Proficient 


35% 


43% 


8th Grade Reading Percent Proficient 


60% 


57% 


8th Grade Math Percent Proficient 


33% 


34% 


8th Grade Science Percent Proficient 


39% 


36% 



For interpretation, Table 5 shows that on the 4 th grade 
reading test, 46% of MPCP 4 th graders for whom the SCDP 
received scores are proficient while 55% of MPS free and 
reduced lunch students are proficient. However, the story 
is again different for 8 th graders. In two of the three subjects, 
MPCP students have higher levels of percent proficient 
than do MPS students. Viewing WKCE scores in terms 
of percent proficient indicate a similar story as described by 
Table 3. In 4 th grade, MPS students tend to score higher on 
the WKCE than do MPCP students; however, the 8 th grade 
results indicate that MPCP students score higher or at least 
as well as their counterparts in MPS. Figures 8 and 9 show a 
graphical representation of the differences between 4 th and 8 th 
grade reading and math respectively. 
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4th and 8th Grade Reading 
Percent Proficiency Comparison 

70% 




4th Grade Reading 8th Grade Reading 

Percent Proficient Percent Proficient 



These are two different sets of students; only a longitudinal study following the same students over time 
would indicate such a finding. 



FIGURE 9 



Figure 9: 4th and 8th Grade Math 
Percent Proficiency Comparison 

50% 




4th Grade Math 8th Grade Math 

Percent Proficient Percent Proficient 



It is tempting to view Figures 8 and 9 as growth by MPCP students from 4 th to 8 th grade; however, such a 
conclusion is tenuous. The data for these two different grades are from two different sets of students. 
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Yet another way to look at the percent proficient data is to aggregate it to the school level. The following 
four figures 12 show the percent proficient in each MPCP school with an indication of the MPS school 
average for that particular grade and subject. 



FIGURE 10 



MPCP 4th Grade Reading 
Percent Proficient by School 




4th Grade Reading Percent Proficient MPS FRL Average 



As indicated in Figure 10, 55% of MPS free and reduced lunch students are proficient in 4 th grade 
reading. Of the 23 MPCP schools for which the SCDP received WKCE 4 th grade reading scores, 10 had 
percentages of proficiency above 55%. Figure 11 shows 8 th grade reading scores in the same manner. 



12 The school averages are not weighted in any way based on the number of students in the school. The minimum 
number of MPCP student scores needed in a grade, subject, and school was 3. 
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MPCP 8th Grade Reading 
Percent Proficient by School 
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8th Grade Reading Percent Proficient MPS FRL Average 



Of the 27 MPCP schools reporting WKCE 8 th grade reading scores, 12 scored higher than the 57% 
average of the MPS schools. 

The figures below showing the math comparison are most telling. Figure 12 indicates that only 8 of the 
22 MPCP schools reporting such scores have a percent proficient above MPS. 



FIGURE 12 

MPCP 4th Grade Math 
Percent Proficient by School 
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Figure 13 indicates the opposite in 8 th grade math with 14 out of 24 schools having percent proficient higher 
than the MPS average. 

FIGURE 13 
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The main point of these particular figures is to indicate the variation of students attending MPCP schools. 
Similar to the WKCE histograms above, MPCP schools serve a range of students with disparate abilities, and 
these charts illustrate well this variation. 



Summary 

The purpose of this report is to provide descriptive data regarding the test scores of Milwaukee Parental 
Choice Program students in grades 4, 8 and 10 in reading, math and science, as reported to the School Choice 
Demonstration Project 2007-2008. The above tables, graphs, and histograms provide a snapshot of these 
students’ percentile scores relative to overall national norms, and scale scores on the WKCE relative to MPS 
FRL students. Because national norms are based on students with “average” educational circumstances, and 
the MPCP exclusively serves low-income inner-city students, the fact that their average levels of performance 
on norm referenced tests cluster around the lower one-third of the distribution is not surprising and should 
not be interpreted to indicate that the MPCP in general or MPCP schools in particular are necessarily doing 
a poor job of educating students. The comparison of the scale scores of the Choice students and schools 
that took the WKCE with the scores of MPS FRL students and schools similarly is presented descriptively 
with no claim that the schools themselves were independently responsible for the various results. Any 
reliable determination of the effectiveness of a school choice program like the MPCP can only come from 
a rigorous longitudinal study that follows a representative group of choice students over time and compares 
their achievement gains to those of a comparable set of public school students. The MPCP Longitudinal 
Education Growth Study (LEGS) will serve as the proper source for such a determination. 
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SCHOOL CHOICE 
DEMONSTRATION PROJECT 

GEORGETOWN UNIVERSITY 



UNIVERSIWARKANSAS 



=» ! -r«l= 



College of Education and Health Professions 
Department of Education Reform 

School Choice Demonstration Project 



201 Graduate Education Building 
Fayetteville, Arkansas 72701 
(479) 575-6345 
(479) 575-3196 (FAX) 



August 21, 2007 



Dear School Administrator: 

It is once again time for testing activities in support of the evaluation of the Milwaukee 
Parental Choice Program. We look forward to your participation in this important study. As 
you may recall, there are two components of the MCPC evaluation that will require your 
participation: program accountability and program evaluation. 

1) Program Accountability 

Participating schools need to test all MPCP students in grades 4, 8, and 10 annually in 
reading, math and, for grades 4 and 8, also in science. Schools can use any nationally- 
normed standardized test. Scores from any standardized testing that the school 
performs must be shared with the School Choice Demonstration Project (SCDP). 

2) Program Evaluation 

To provide a more rigorous evaluation of the MPCP, a representative sample of MPCP 
students in grades 3-8 and 10 will be tested using the WKCE. Students in grade 9 will be 
administered the Milwaukee Benchmark Assessment. 

Schools that administer the WKCE comply with both components using the same test. If 
you plan to administer the WKCE this fall, any MPCP panel students who are already taking 
the WKCE at your school will not need to be double tested to fulfill the program evaluation 
requirement. If your school plans to administer the WKCE to only a few grades (e.g. grades 
4 and 8 only), then SCDP will rely on your school to separately test any MPCP panel students 
in the remaining grades (3-8 and 10) and then share the scores with us, the research team. 

If your school will be administering the WKCE fill out the box located on the 
School Contact Form. Please indicate which grades will be tested as well as the 
dates when testing will take place. 

Test administrators in all grades where panel students have been selected (except 9th grade) 
are required to attend the WKCE training. Please include the names of all staff who will be 
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administering the WKCE and the grade(s) in which they will be administering the test on 
the enclosed Test Training Registration Form. For each test administrator, check the 
appropriate box to indicate the training session he/she will attend. Local staff trained in test 
administration will be available to assist you in testing panel students in all grades at your 
school. This may be very useful for testing students who are absent on the day of testing 
or any other special testing circumstance. 

Laura Collins of our study team will coordinate directly with your designated School Contact 
(the person noted on the School Contact Form) to work out the logistics for testing. She 
will work with your School Contact to arrange local staff needed for test administration. 
Later, she will provide guidelines on submitting test scores to the study team. Her contact 
information is as follows: 

Laura Collins 

Westat RA1217 

1650 Research Blvd 

Rockville, MD 20850 

Direct Line: 301-610-5573 

Toll Free: 1-800-937-8281 x5573 

Fax: 301-294-4475 

Email: lauracollins@westat.com 

Please feel free to contact Laura Collins directly if you have any questions. 



Sincerely, 




Patrick J. Wolf, Ph.D. 

Professor and Endowed Chair in School Choice 
Principal Investigator, SCDP 
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SCHOOL CHOICE 
DEMONSTRATION PROJECT 

GEORGETOWN UNIVERSITY 



UNIVERSITY^ARKANSAS 

=»i;ri»= a 



College of Education and Health Professions 201 Graduate Education Building 

Department of Education Reform Fayetteville, Arkansas 72701 

School Choice Demonstration Project (479) 575-6345 

(479) 575-3196 (FAX) 



The Longitudinal Evaluation of the Milwaukee Parental Choice Program 
Frequently Asked Questions 

Why do we have to test MPCP students? 

The testing requirements in the Milwaukee Parental Choice Program (MPCP) are the result of 2005 Act 
125 passed by the state legislature and signed into law on March 10, 2006. The new law also raises the 
cap on total participation in the MPCP to 22,500 students and requires that all schools participating in the 
MPCP obtain accreditation, among other changes. 

Act 125 outlines the two components of the required testing program: program accountability and 
program evaluation. MPCP schools have vital responsibilities in both of these areas. 

COMPONENT ONE: PROGRAM ACCOUNTABILITY 

1.1 Who and what are we required to test for program accountability purposes? 

Under the new law, all MPCP students in grades 4, 8, and 10 must be tested annually. The test must 
cover reading, math, and science. 

1.2 What test can we use? 

Schools can use any nationally-normed standardized test. Qualified tests would include the Iowa Test of 
Basic Skills, SAT 10, Terra Nova, or the Wisconsin Knowledge and Concepts Exam (WKCE). 

1.3 Who pays for the test? 

The participating MPCP schools are expected to pay for the costs of purchasing and scoring the test that 
they choose to administer. 

1.4 When should we test? 

Schools that choose to administer the WKCE for accountability purposes need to test during the fall 
testing window of October 22nd through November 23rd . Schools that choose to administer a national 
norm-referenced test can do so in the fall or spring. 

1.5 What testing information do we need to provide to whom for program accountability? 

Act 125 requires that, beginning in 2006, all MPCP schools forward to the School Choice Demonstration 
Project (SCDP) the individual level results of all standardized tests that it administers. That means that 
if the school tests all students in grades 4, 8, and 10 - not just MPCP students - the scores for all those 
students must be provided to the SCDP. Also, if a participating MPCP school administers standardized 
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tests to students in grades besides 4, 8, and 10, the results of those tests must also be forwarded 
to the SCDP. The basic requirements of Act 125 regarding testing are that participating schools 
must test all 4th, 8th, and 10th grade MPCP students annually, can test non-MPCP students and 
MPCP students in other grades, and must forward to the SCDP the individual-level results of 
any standardized testing that they conduct. 

1.6 Why is testing information being provided to the SCDP? 

Through Act 125, the state government of Wisconsin has ordered the SCDP to conduct a 
series of data collection, analysis, and reporting activities focused on evaluating the MPCP as 
a whole. The data collection activities include collecting the test scores from all standardized 
testing conducted by participating MPCP schools and converting those scores into data that 
can be compared both with MPS test scores and over time. The SCDP also is required to draw 
a representative sample of MPCP students and obtain test scores from them on the WKCE 
in order to make it easier to compare MPCP performance with MPS performance. These 
comparisons and evaluations will be made very carefully, by an experienced team of school 
choice researchers. Under the law, the data and results of the evaluations will be forwarded 
to the Legislative Audit Bureau for secondary analysis and review, and reported to the state 
legislature. 

1.7 What specific testing information do we need to provide to the SCDP? 

The SCDP evaluation team will need the following information about each student that MPCP 
schools test: 

• At least two measures of the student’s scores in the three sections (reading, math, 
and science). Scale scores are preferred, but Normal Curve Equivalents (NCEs) and 
National Percentile Ranks (NPRs) are also acceptable. For example, a school might 
provide each student’s scale scores and NCEs in reading, math, and science. Raw 
scores will not be useful to the evaluators. 

• Basic information about the test that was administered and the student who produced 
the scores, including the name of the test, the date(s) of testing, if any special 
accommodations were made for the student, as well as the name, grade level, birthday 
and school attended by the student. In many cases, all or most of this information will 
be on the individual student records that the testing company provides to schools upon 
scoring the tests. 

1.8 In what form should we provide that information to the SCDP? 

If your school receives the results of its testing program in electronic format, as a database, 
then we strongly prefer that you forward the information to us electronically. If you receive the 
results in paper form, please make copies of the paper records and send them to us. 

1.9 How should we transmit this information to the SCDP? 

Because the testing information will include sensitive personal information about your students, 
you will need to take special precautions in providing us with the data. One option is for you 
to arrange for SCDP personnel to pick up the testing information in person. Once you have 
received and made a copy of your testing database or paper records, just call our University of 
Arkansas office and we will make arrangements for a member of our research staff to come by 
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your office to obtain the information. A second option is for you to ship the information to the University 
of Arkansas office of the SCDP using a secure method of shipping (e.g. Fedex, UPS, or registered U.S. 
Mail). 

1.10 Will the names of students and schools be kept confidential? 

Absolutely. This information is being collected in order to evaluate the MPCP as a whole and to learn 
more about the effects of school choice programs like the MPCP. No data will be reported at the 
individual level, and no names of participating individuals or schools will ever be reported to anyone 
outside of the SCDP research team. This research is being overseen by the Institutional Review Board 
(IRB) for Fluman Subjects Research at the University of Arkansas. The IRB requires that the identity 
of all participants - individuals and schools - be kept strictly confidential in the conduct of academic 
research such as the MPCP longitudinal evaluation. As a result of the study, the public will learn how 
well the choice program is performing, and perhaps how the program might be improved, but no one will 
be given any specific information about the students and schools in the MPCP. The SCDP has strong 
safeguards in place to make sure that complete confidentiality is maintained throughout the study. 

COMPONENT TWO: PROGRAM EVALUATION 

2.1 In addition to the testing for program accountability purposes, what else are private schools 
participating in the MPCP expected to do in support of the study? 

The SCDP research team will need help from private schools in conducting the MPCP impact study 
that is called for in the law, based on the representative sample of MPCP students. Last fall we drew 
a representative sample of approximately 2,500 MPCP students in grades 3-8 and the entire census of 
MPCP 9th graders. This October we will draw an additional representative sample of 500 3rd graders. 

All panel students will be administered the same tests taken by their respective public school peers (e.g. 
the WKCE-CRT in grades 3-8 and 10 and the Milwaukee Benchmark Assessment in grade 9). The testing 
of the “MPCP Panel” will take place sometime between October 22nd and November 23rd to coincide 
with MPS testing. We will follow this subgroup of MPCP students for another four years, re-testing them 
annually and surveying them and their parents about their educational experiences. The research team 
will handle all of the survey activities, but we will need help from the MPCP schools in testing the MPCP 
panel. We will notify schools early this fall regarding which of then MPCP students have been selected 
for the MPCP panel, and then work with the schools to arrange for the administration of the WKCE-CRT 
and the Benchmark Assessment to panel students during the fall testing window. 

2.2 Who will administer the WKCE-CRT to the students on the MPCP panel and how will they do it? 

We prefer that the WKCE-CRT be administered to students on the MPCP panel in their own school by 
school personnel. Such testing conditions would best replicate how MPS students are tested and therefore 
would facilitate a proper comparison between the performance of MPCP and MPS students. The research 
team will purchase these tests and pay for their scoring. We also will provide training on WKCE test 
administration to the school personnel assigned the responsibility for conducting the testing (training 
is not necessary for the Milwaukee Benchmark Assessment). Since it is highly unlikely that the MPCP 
panel will include all 3-10 grade students in a particular school, or even all such MPCP students in a 
particular school, we recommend that the testing of the MPCP panel students be conducted on a “pull out” 
basis in order to minimize disruption of the school schedule, and be conducted over several days so that 
students are not spending an entire day away from their classmates and the curriculum. 
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Since the entire census of MPCP 9th graders was selected to participate in the panel last fall, schools 
should expect that close to all of their MPCP 10th graders will be on the panel this year. If a school has 
a few 10th graders that were not part of the panel last year, but does not wish to exclude those students 
from testing this year, we will provide materials to test all 10th graders. 

Schools that lack the personnel resources to conduct the testing themselves should contact the Westat 
representative on the SCDP to request assistance. SCDP staff will be available throughout the testing 
period to provide advice, support, and quality control over the test administrations. As soon as the 
testing of the MPCP panelists at a particular school is complete, administrators should contact Westat 
and a staff member will come by to pick up the testing materials for processing. 

2.3 Why are there two testing requirements instead of one? Can the two requirements be satisfied 
by a single testing program? 

There are two testing requirements as a result of Act 125 because one is appropriate for program 
accountability purposes and the other is appropriate for program evaluation purposes. Participating 
schools are required to test all of their MPCP students in grades 4, 8, and 10 using the qualified test 
they choose in order to generate program-wide information about how well MPCP students are doing. 
That information could not serve as the foundation for a reliable study of the effects of the MPCP 
program on student outcomes because the sample is limited to a few grades and the students will be 
assessed using a variety of tests. In order to determine if the MPCP program itself is improving student 
outcomes, we need to follow a representative panel of MPCP students and assess their performance 
using the same test that comparable MPS students take - the WKCE-CRT. In specific schools, the 
group of students that take the tests, and the specific tests that are administered, will likely differ for 
purposes of accountability (all 4th, 8th, and 10th MPCP students on school- selected test) and evaluation 
(a sample of 3rd- 10th MPCP students on the MPS test). The two testing programs will only perfectly 
overlap at a particular school if both of the following conditions hold: 

• All MPCP students in grades 3-8 (if elementary) or 9 and 10 (if high school) are tested in 
reading, math, and science in the fall of every year; 

• The school administers the public school test (WKCE-CRT if grades 3-8 and 10, Milwaukee 
Benchmark Assessment if grade 9) as its school-selected test. 

Under those conditions, the school could simply provide the SCDP with a copy of the results of its 
annual testing program and the school’s testing requirements under Act 125 would be completely 
satisfied. Some participating MPCP schools might administer the WKCE-CRT as their annual student 
test, but not necessarily to the students in all of the grades 3-8. If that is the case, we will accept the 
test scores of the students that were tested and make arrangements with the school to test any students 
in the MPCP panel attending that school that were not tested. Thus, for private schools that administer 
the WKCE-CRT, there will be overlap between the two testing programs but the overlap might not be 
complete. 
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2.4 What testing accommodations should we provide for students with special needs? 

We urge test administrators to provide appropriate testing accommodations based on the type and 
severity of special needs that students may have. In the public schools, such accommodations 
are detailed in a students Individualized Education Plan (IEP). Although IEPs are not usually 
used by private schools, if a specific is known to have or to have had an IEP that specified testing 
accommodations, then those accommodations should be followed for purposes of MPCP testing 
to the extent possible. In most cases, MPCP students with disabilities or language challenges that 
affect their ability to take tests will not have IEPs as a guide. In those cases, school personnel 
must do the following: 

• Notify Westat regarding who will be accommodated, how, and why; 

• Document all accommodations on a sheet sent to the test scoring company along with the 
testing materials (Westat will provide one if the testing company does not); 

• Administer the same set of accommodations to the same students each year that they are 
tested. 

The state of Wisconsin administers alternate assessments to students with disabilities (the 
WAA-SwD) and for English language learners (WAA-ELL) in the same subjects by grade as 
the WKCE-CRT. Westat will have copies of those alternative assessments available for testing 
purposes and will train test administrators on how to use them. 

CONTACT INFORMATION 

What if we have additional questions regarding the testing requirements under Act 125? 

For questions regarding what participating MPCP schools are required to do under the new law, 
please contact the Department of Public Instruction: 

Tricia Collins: 608-266-2853 or toll free 1-888-245-2732, ext 3 then 3 

Lisa Geraghty: 608-266-0523 or toll free 1-888-245-2732, ext 3 then 5 



For questions regarding the specific process of transferring the results of your school’s regular 
testing program to the School Choice Demonstration Project, please contact the University of 
Arkansas: 

Laura Jensen: 479-575-6345 

For questions regarding the specific process for testing students on the MPCP panel, please 
contact Westat: 

Laura Collins: 301-610-5573 

Sabria Hardy: 301-610-5513 
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Exhibit 3 




SCHOOL CHOICE 
DEMONSTRATION PROJECT 

GEORGETOWN UNIVERSITY 



UNIVERSITOARKANSAS 






College of Education and Health Professions 
Department of Education Reform 

School Choice Demonstration Project 



201 Graduate Education Building 
Fayetteville, Arkansas 72701 
(479) 575-6345 
(479) 575-3196 (FAX) 



July 15, 2008 



Dear School Administrator: 

I hope this letter finds you doing well as you are wrapping up the end of what I trust 
was a successful academic year. I want to begin by thanking you for all your assistance 
throughout this year as we gathered data for the second year of the state-mandated Lon- 
gitudinal Evaluation of the Milwaukee Parental Choice Program. 

According to our records, your school has not mailed us your principal survey or your 
2007-08 student test scores. I have included an additional copy of the principal survey 
for your use. As a reminder we need a copy of all individual-level test results from ANY 
standardized testing that your school did during this school year. Please use the enclosed 
FedEx package to send us any outstanding documents. 

As always, if you have any questions, please do not hesitate to give Laura Jensen a call 
(479-575-6345). I wish you the best of luck with your end of the school year activities. 



Sincerely, 




fh.D. 



Patrick Wolf, Th.D. 

Principal Investigator 

School Choice Demonstration Project 
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SCHOOL CHOICE 
DEMONSTRATION PROJECT 

GEORGETOWN UNIVERSITY 



UN1VERSIWARKANSAS 






College of Education and Health Professions 
Department of Education Reform 

School Choice Demonstration Project 



201 Graduate Education Building 
Fayetteville, Arkansas 72701 
(479) 575-6345 
(479) 575-3196 (FAX) 



February 5th, 2008 



Dear School Administrator: 

As our previous correspondence has mentioned, Wisconsin Act 125 mandates that all schools 
participating in the Milwaukee Parental Choice Program (MPCP) annually administer a nationally 
normed, standardized test of their choosing or the WKCE to their choice students in grades 4, 8, and 
10. The test must cover reading, math and science. Additionally, Act 125 requires that MPCP schools 
submit a copy of all individual student level scores from any standardized tests they administer to the 
School Choice Demonstration Project (SCDP) at the University of Arkansas. The purpose of this letter 
is to provide you with guidance regarding how to submit your school's test scores to the SCDP. 

Once you have received the scores from all your 2007-08 testing, please copy the individual student 
level score sheets and place the copies in the FedEx package you received along with this letter. You 
will also find enclosed in this package a copy of the MPCP Principals' Survey. Please complete this 
survey and insert it in the FedEx package along with your school's test scores. Please complete the 
top portion of the FedEx mailing form and remove the top copy for your records. This top copy can be 
used to track the package in the event that it does not arrive at the SCDP. 

Time Frame & Mailing Address 

Results of any fall testing that was administered at your school should be mailed to the SCDP by 
March 1st. Results of any spring testing that your school administered should be mailed to the SCDP 
by July 1st. Test score information should be mailed to the following address: 

Laura Jensen 

School Choice Demonstration Project 
Department of Education Reform 
201 Graduate Education Building 
Fayetteville, AR 72701 
479-575-6345 

Format of Scores 

A copy of the scores from any standardized test (e.g. Terra Nova, ITBS, WKCE, etc.) administered 
at your school can be submitted to the SCDP in either electronic or paper format, though electronic 
format is strongly preferred. 
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Security Protocols 

To protect the confidentiality of your students, it is important certain data protection strategies be 
implemented. If you are sending an electronic copy of your students' scores, you must password protect 
the file and burn the file to a CD (see attached sheet with instructions on CD burning). Instructions on 
how to password protect Microsoft Word, Microsoft Excel, and Acrobat Professional documents can be 
found below. 

Securing a Word or Excel file with a password: 

With the file open: 

On the Tools menu, click Options, and then click Security. 

In the Password to open box, type MPCP2006, and then click OK. 

In the Reenter password to open box, type MPCP2006 again, and then click OK. 

Securing an Acrobat Professional/PDF file with a password: 

With the PDF document open: 

Control D for Document Properties 

Click Security Tab > Change Security Method to Password Security 
Check "Require a password to open the document" 

Enter a password MPCP2006 in the "Document Open Password:” box 
Confirm the password MPCP2006 

If the electronic file containing your students' scores is not in any of these formats, please check the help 
menu in the program that fits the file for guidance in how to password protect the file. Another option 
would be to convert the file from its program format into Excel, which is the most preferred format for 
transferring data files securely. 

If you chose to submit a paper copy of your students' test scores, the package in which your documents 
are mailed must be sealed with the mailer's signature over a taped seal. When mailing your scores, make 
sure that you indicate that a signature of the addressee is required at the point of destination (i.e. School 
Choice Demonstration Project). Paper copies of scores can be mailed securely by Registered U.S. Mail, 
Federal Express, UPS, etc. 

We look forward to receiving your school's test scores. As always, if you have any questions, please call 
Laura Jensen, SCDP Research Assistant, at 479-575-6345. 

Sincerely yours, 




Patrick J. Wolf Ph.D. 

Professor and Endowed Chair in School Choice 
Principal Investigator, SCDP 
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A ppendix A: 

Schools with MPCP Students in Tested Grades Operating through May 2008 
N=1 1 5; (The italicized school did not provide test scores in 2007-08) 



Agape Center of Academic Excellence, Inc. 


Eastbrook Academy 


Messmer High School 


Alston's Preparatory Academy 


Emmaus Lutheran School 


Messmer Prep Catholic School 


Atlas Preparatory Academy 


Excel Academy 


Milwaukee Lutheran High School 


Atonement Lutheran School 


Excel Learning Academy 


Milwaukee Montessori School 


Believers in Christ Christian Academy 


Fairview Lutheran School 


Milwaukee Seventh Day Adventist School 


Bessie M. Gray Prep Academy 


Family Montessori School 


Mother of Good Counsel School 


Blessed Sacrament School 


Garden Homes Community Montessori 
School, Inc. 


Mount Calvary Lutheran School 


Blessed Savior - East Campus 


Garden Homes Lutheran School 


Mount Lebanon Lutheran 


Blessed Savior - North Campus 


Gilchrist Christian Academy 


New Testament Christian Academy 


Blessed Savior - West Campus 


Gospel Lutheran School 


Northwest Lutheran School 


Blessed Savior - South Campus 


Greater Holy Temple Christian Center 


Notre Dame Middle School 


Blyden Delany Academy 


Harambee Community School 


Oklahoma Avenue Lutheran School 


Carter's Christian Academy 


Hickman Academy Preparatory School 


Our Lady of Good Hope School 


Catholic East Elementary School 


Holy Redeemer Christian Academy 


Our Lady Queen of Peace Parish 


CEO Leadership Academy 


Holy Wisdom Academy 


Parklawn Christian Leadership Academy 


Ceria M. Travis Academy, Inc. 


Hope Christian School 


Pius XI High School 


Christ Memorial Lutheran School 


Hope Middle School 


Prince of Peace 


Christ St. Peter Lutheran School 


Institute for Career Empowerment Inc. 


Resurrection Christian Academy 


Christian Faith Academy of Higher Learning 


Institute of Technology and Academics 


Right Step, Inc. 


Clara Mohammed School 


Jared C. Bruce Academy 


Risen Savior Lutheran School 


Community Vision Academy LTD 


Jesus Academy of Learning 


Saint Adalbert School 


Concordia University School and Institute 


Saint Anthony School 


for LIGHT 


Johnson Christian Academy, Inc. 


Saint Bernadette School 


Daughters of the Father Christian Academy 


KidPreneur 


Saint Catherine of Alexandria 


Destiny High School 


King's Academy Christian School 


Saint Catherine School 


Divine Savior Holy Angels High School 


La Brew Troopers Military University School 


Saint Charles Borromeo School 


Dr. Brenda Noach Choice School 


Lutheran Special School & Education 




Early View Academy of Excellence 


Services 


Saint Gregory the Great Parish School 


Marquette University High School 


Saint Joan Antida High School 
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Saint John Kanty School 
Saint John's Evangelical Lutheran 
Saint Josaphat Parish School 
Saint Leo Catholic Urban Academy 
Saint Lucas Lutheran School 
Saint Marcus Lutheran School 
Saint Margaret Mary School 
Saint Martini Lutheran School 
Saint Peter-lmmanuel Lutheran School 
Saint Philip's Lutheran School 
Saint Rafael the Archangel School 
Saint Roman Parish School 
Saint Rose Catholic Urban Academy 



Saint Sebastian School 

Saint Thomas Aquinas Academy 

Saint Vincent Pallotti School 

Salam School 

Sharon Junior Academy 

Sherman Park Lutheran School/Preschool 

Siloah Lutheran School 

Tamarack Community School 

Texas Bufkin Academy 

The Hope School 

Travis Technology High School 

Trinity Christian Academy for Nonviolence 

Urban Day School 



Victory Christian Academy 

Victory Preparatory Academy 

Washington DuBois Christian Leadership 
Academy 

Wisconsin Lutheran High School 
Word of Life Evangelical Lutheran School 
Yeshiva Elementary School 
Young Minds Preparatory School 



MPCP Schools with no MPCP Students in Tested Grades 2007-2008 

N=9 



CrossTrainers Academy 
Family Academy 

Grace Preparatory School of Excellence 

Kindergarten Plus 

Malaika Early Learning Center 



Mustard Seed International School 

New Holy Ghost Tabernacle Pre-School/ 
Academy 

Paige II University School, Inc. 



The AppleCrest Preparatory Leadership 
Academy 
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